Projection of user interface pose command to reduced degree of freedom space for a surgical robot

ABSTRACT

For teleoperation of a surgical robotic system, the user command for the pose of the end effector is projected into a subspace reachable by the end effector. For example, a user command with six DOF is projected to a five DOF subspace. The six DOF user interface device may be used to more intuitively control, based on the projection, the end effector with the limited DOF relative to the user interface device.

BACKGROUND

The present embodiments relate to robotic systems for minimally-invasivesurgery (MIS). MIS may be performed with robotic systems that includeone or more robotic arms for manipulating surgical tools based oncommands from a remote operator. A robotic arm may, for example, supportat its distal end various surgical end effectors, including staplers,scalpels, imaging devices (e.g., endoscope), clamps, and scissors. Usingthe robotic system, the surgeon controls the robotic arms and endeffectors in teleoperation during MIS.

Different end effectors, in combination with the robotic arms, havedifferent numbers of degrees of freedom (DOF). For example, a staplerhas five DOF in teleoperation. Five active joints, and correspondingDOF, includes two joints on the instrument (e.g., rotation andarticulate to control yaw) and three joints on the robotic arm (e.g.,spherical rotate, spherical pitch, and tool translate). The end effectoris positioned and oriented with only five DOF. In yet another example,an endoscope or ultrasound scalpel has four DOF in teleoperation. Fouractive joints, and corresponding DOF, include one joint on theinstrument (e.g., rotation) and three joints on the robotic arm (e.g.,spherical rotate, spherical pitch, and tool translate).

During teleoperation, the user inputs using a user input device with agreater number of DOF, such as six DOF. These user pose commands inCartesian space are translated into joints motions, such that theinstrument follows the user commands or inputs with controlled accuracy.Tools having fewer DOFs than the input do not track the more arbitraryspatial commands (position and orientation) from the user input. Thiscan lead to undesirable or un-intuitive behavior in the joint commandswhere the commanded motion is in a direction that is not feasible forthe robot. Where the user input is ungrounded (e.g., 6 DOF), the motionof the user input device cannot be restricted by mechanical fixtures orhaptic feedback to prevent the user from inputting motion for which theend effector is not capable.

SUMMARY

By way of introduction, the preferred embodiments described belowinclude methods, systems, instructions, and computer readable media forteleoperation of a surgical robotic system. The user input for the poseof the end effector is projected into a subspace reachable by the endeffector. For example, a user input with six DOF is projected to a fiveDOF subspace. The six DOF user interface device may be used to moreintuitively control, based on the projection, the end effector with thelimited DOF relative to the user interface device.

In a first aspect, a method is provided for teleoperation of a surgicalrobotic system. A user input to move a surgical tool coupled to arobotic arm is received. The user input has six degrees of freedom wherethe surgical tool has a lesser number of degrees of freedom. The userinput is projected to the lesser number of degrees of freedom. Jointmotion of the robotic arm and surgical tool is solved from the projecteduser input with inverse kinematics. The robotic arm and/or surgical toolare moved based on a solution from the solving.

In a second aspect, a method is provided for accounting for a limiteddegree of freedom of a tool in a surgical robotic system. A first posefrom an ungrounded user interface with six degrees of freedom isprojected to a second pose of an end effector of a surgical tool held bya robotic arm. The second pose has only four or five degrees of freedom.The end effector is controlled based on the second pose.

In a third aspect, a surgical robotic system is provided. A surgicalinstrument is mountable to a robotic arm. The surgical instrument has anend effector where rotation about one axis is coupled to rotation aboutanother axis. A user interface device has three degrees of freedom inrotation. A controller is configured to project a user command from theuser interface device for rotation about the one axis to rotations aboutthe one axis and the other axis.

The present invention is defined by the following claims, and nothing inthis section should be taken as a limitation on those claims. Anyteaching for one type of claim (e.g., method) may be applicable toanother type of claim (e.g., computer readable storage medium orsystem). Further aspects and advantages of the invention are discussedbelow in conjunction with the preferred embodiments and may be laterclaimed independently or in combination.

BRIEF DESCRIPTION OF THE DRAWINGS

The components and the figures are not necessarily to scale, emphasisinstead being placed upon illustrating the principles of the invention.Moreover, in the figures, like reference numerals designatecorresponding parts throughout the different views.

FIG. 1 is an illustration of one embodiment of an operating roomenvironment with a surgical robotic system according to one embodiment;

FIG. 2 illustrates an example surgical robot arm and surgical tool;

FIG. 3 is a flow chart diagram of one embodiment of a method forteleoperation of a surgical robotic system;

FIG. 4 shows some example frames of reference for a robotic arm andsurgical tool;

FIG. 5 is a block diagram of one embodiment of a surgical roboticsystem.

DETAILED DESCRIPTION OF THE DRAWINGS AND PRESENTLY PREFERRED EMBODIMENTS

A pose command from a user interface device is projected to a reducedDOF space. For example, 6D pose commands from an ungrounded userinterface device are projected by a program into the reachable subspaceduring teleoperation. To achieve arbitrary end effector positions andorientations in 6D, the robot manipulator must have six actuated DOF orjoints. Thus, for teleoperation with a surgical instrument having morelimited DOF (e.g. a stapler instrument), the 6D pose commands areprojected into a subspace for the more limited DOF (e.g., a 5D or 4Dsubspace) that is reachable by the robot end effector.

Different subspaces may be used for the projection. The DOF for thesubspace may be any combination of translation and rotation (e.g.,articulation and orientation) with the limited DOF, such as translationwith 3 DOF and rotation with 2 DOF. There is more than one possiblechoice for the subspace (e.g., 4D or 5D subspace) onto which to projectthe 6D pose command. The choice of projection method may affect theuser's ease of controlling the surgical instrument. The user is able tomove the user interface device more freely in position and orientation(e.g., 6D), but the end effector will only move in the subspaceaccording to the projection method. Thus, motion in one or more DOFdirection is lost or limited, making it more difficult for the user toplace the end effector position and orientation at a desiredconfiguration. In one embodiment, the subspace has only 1 or 2 DOF ofrotation and 3 DOF for translation to make the user interface asintuitive as possible. Other subspaces may be used.

FIGS. 1 and 2 show an example surgical robotic system. The approachesfor projection of the user inputs or commands for the end effector to alower dimensional subspace are discussed below in reference to thisexample system. Other surgical robotic systems and surgical robots ornon-surgical robotic systems and robots may use the projection to reducedimensionality of the user command for pose or change in pose of the endeffector.

FIGS. 3 and 4 are directed to projection of user inputs or commands incommand space in teleoperation. FIG. 6 is directed to a system for usingthe projection to form a projected command, which is used in the inversekinematics with a medical robotic system for teleoperation.

FIG. 1 is a diagram illustrating an example operating room environmentwith a surgical robotic system 100 for which commands from the user areconverted into motion of the surgical robotic arms 122 with inversekinematics. The surgical robotic system 100 includes a user console 110,a control tower 130, and a surgical robot 120 having one or moresurgical robotic arms 122 mounted on a surgical platform 124 (e.g., atable or a bed etc.), where surgical tools with end effectors areattached to the distal ends of the robotic arms 122 for executing asurgical procedure. Additional, different, or fewer components may beprovided, such as combining the control tower 130 with the console 110or surgical robot 120. The robotic arms 122 are shown as table-mounted,but in other configurations, the robotic arms 122 may be mounted in acart, a ceiling, a sidewall, or other suitable support surfaces.

Generally, a user, such as a surgeon or other operator, may be seated atthe user console 110 to remotely manipulate the robotic arms 122 and/orsurgical instruments (e.g., teleoperation). The user console 110 may belocated in the same operation room as the robotic system 100, as shownin FIG. 1 . In other environments, the user console 110 may be locatedin an adjacent or nearby room, or tele-operated from a remote locationin a different building, city, or country. The user console 110 mayinclude a seat 112, pedals 114, one or more handheld user interfacedevices (UIDs) 116, and an open display 118 configured to display, forexample, a view of the surgical site inside a patient and graphic userinterface. As shown in the exemplary user console 110, a surgeon sittingin the seat 112 and viewing the open display 118 may manipulate thepedals 114 and/or handheld user interface devices 116 to remotely anddirectly control the robotic arms 122 and/or surgical instrumentsmounted to the distal ends of the arms 122. The user inputs commands forthe movement of the surgical arms 122 and/or end effectors. This usercontrol determines pose (position and orientation) of the robotic arms122. The surgeon sitting in the seat 112 may view and interact with thedisplay 118 to input commands for movement in teleoperation of therobotic arms 122 and/or surgical instruments in the surgery.

In some variations, a user may also operate the surgical robotic system100 in an “over the bed” (OTB) mode, in which the user is at thepatient's side and simultaneously manipulating a robotically-driventool/end effector attached thereto (e.g., with a handheld user interfacedevice 116 held in one hand) and a manual laparoscopic tool. Forexample, the user's left hand may be manipulating a handheld userinterface device 116 to control a robotic surgical component while theuser's right hand may be manipulating a manual laparoscopic tool. Thus,in these variations, the user may perform both robotic-assisted MIS andmanual laparoscopic surgery on a patient.

During an exemplary procedure or surgery, the patient is prepped anddraped in a sterile fashion to achieve anesthesia. Initial access to thesurgical site may be performed manually with the robotic system 100 in astowed configuration or withdrawn configuration to facilitate access tothe surgical site. Once the access is completed, initial positioningand/or preparation of the robotic system may be performed. During theprocedure, a surgeon in the user console 110 may utilize the pedals 114and/or user interface devices 116 to manipulate various end effectorsand/or imaging systems to perform the surgery using teleoperation. Themovements may be surgeon, patient, and/or situation specific, so mayvary. Manual assistance may also be provided at the procedure table bysterile-gowned personnel, who may perform tasks including but notlimited to, retracting tissues or performing manual repositioning ortool exchange involving one or more robotic arms 122. Some surgicaltasks, such as retracting, suturing, stapling, or other tissuemanipulation, may instead be performed by one or more robotic arms 122(e.g., third or fourth arms). Nonsterile personnel may also be presentto assist the surgeon at the user console 110. When the procedure orsurgery is completed, the robotic system 100 and/or user console 110 maybe configured or set in a state to facilitate one or more post-operativeprocedures, including but not limited to, robotic system 100 cleaningand/or sterilization, and/or healthcare record entry or printout,whether electronic or hard copy, such as via the user console 110.

In some aspects, the communication between the surgical robot 120 andthe user console 110 may be through the control tower 130, which maytranslate user input commands from the user console 110 to roboticcontrol commands and transmit the control commands to the surgical robot120. The control tower 130 performs inverse kinematics. The controltower 130 may also transmit status and feedback from the robot 120 backto the user console 110. The connections between the surgical robot 120,the user console 110, and the control tower 130 may be via wired and/orwireless connections and may be proprietary and/or performed using anyof a variety of data communication protocols. Any wired connections maybe optionally built into the floor and/or walls or ceiling of theoperating room. The surgical robotic system 100 may provide video outputto one or more displays, including displays within the operating room,as well as remote displays accessible via the Internet or othernetworks. The video output or feed may also be encrypted to ensureprivacy and all or portions of the video output may be saved to a serveror electronic healthcare record system.

Prior to initiating surgery with the surgical robotic system, thesurgical team can perform preoperative setup. During the preoperativesetup, the main components of the surgical robotic system (e.g., table124 and robotic arms 122, control tower 130, and user console 110) arepositioned in the operating room, connected, and powered on. The table124 and robotic arms 122 may be in a fully-stowed configuration with thearms 122 under the table 124 for storage and/or transportation purposes.The surgical team can extend the arms 122 from their stowed position forsterile draping. After draping, the arms 122 can be partially retracteduntil needed for use. A number of conventional laparoscopic steps mayneed to be performed including trocar placement and insufflation. Forexample, each sleeve can be inserted with the aid of an obturator, intoa small incision and through the body wall. The sleeve and obturatorallow optical entry for visualization of tissue layers during insertionto minimize risk of injury during placement. The endoscope is typicallyplaced first to provide hand-held camera visualization for placement ofother trocars. After insufflation, if required, manual instruments canbe inserted through the sleeve to perform any laparoscopic steps byhand.

Next, the surgical team may position the robotic arms 122 over thepatient and attach each arm 122 to a corresponding sleeve. The surgicalrobotic system 100 has the capability to uniquely identify each tool(endoscope and surgical instruments) upon attachment and display thetool type and arm location on the open or immersive display 118 at theuser console 110 and the touchscreen display on the control tower 130.The corresponding tool functions are enabled and can be activated usingthe master UIDs 116 and foot pedals 114. The patient-side assistant canattach and detach the tools, as required, throughout the procedure. Thesurgeon seated at the user console 110 can begin to perform surgery asteleoperation using the tools controlled by two master UIDs 116 and footpedals 114. The system translates the surgeon's hand, wrist, and fingermovements through the master UIDs 116 into precise real-time movementsof the surgical tools. Therefore in direct teleoperation, the systemconstantly monitors every surgical maneuver of the surgeon and pausesinstrument movement if the system is unable to precisely mirror thesurgeon's hand motions. The UIDs 116 may move in six DOF, such asallowing translation in three dimensions and rotation about the threedimensions. The foot pedals 114 may be used to activate various systemmodes, such as endoscope control and various instrument functionsincluding monopolar and bipolar cautery, without involving surgeon'shands removed from the master UIDs 116.

FIG. 2 is a schematic diagram illustrating one exemplary design of arobotic arm, a tool drive, and a cannula loaded with a robotic surgicaltool, in accordance with aspects of the subject technology. As shown inFIG. 2 , the example surgical robotic arm 122 may include a plurality oflinks (e.g., a link 204) and a plurality of actuated joint modules(e.g., a joint 202, see also joints J1-8) for actuating the plurality oflinks relative to one another. The joint modules may include varioustypes, such as a pitch joint or a roll joint, which may substantiallyconstrain the movement of the adjacent links around certain axesrelative to others.

Also shown in the exemplary design of FIG. 2 is a tool drive 210attached to the distal end of the robotic arm 122. The tool drive 210may include a cannula 214 coupled to its end to receive and guide asurgical instrument 220 (e.g., endoscopes, staplers, scalpel, scissors,clamp, retractor, etc.). The surgical instrument (or “tool”) 220 mayinclude an end effector 222 at the distal end of the tool. The pluralityof the joint modules of the robotic arm 122 can be actuated to positionand orient the tool drive 210, which actuates the end effector 222 forrobotic surgeries. The end effector 222 is at a tool shaft end and/orlast joint (e.g., articulation joint J10). In other embodiments, thetool shaft end is a tip of a needle or other object.

In the example of FIG. 2 , the joint J0 is a table pivot joint andresides under the surgical table top. Joint J0 is nominally held inplace during surgery and may be used as a reference frame or base framefor operation of the robotic arm 122. Joints J1 to J5 form a setup orCartesian arm and are nominally held in place during surgery orteleoperation, so do not contribute to motion during surgicalteleoperation. Joints J6 and J7 form a spherical arm that may activelymove during surgery or teleoperation. Joint J8 translates the tool 220,such as the end effector 222, as part of a tool driver. Joint J8 mayactively move during surgery. Joint J9 rotates the longitudinal shaft ofthe tool 220 about the longitudinal axis. Joint J10 is a wrist on thetool 220, such as a wrist that rotates about one axis perpendicular tothe longitudinal shaft.

Joints J6-10 actively position a tool shaft end (i.e., end effector 222)during surgery while maintaining an entry point into the patient at afixed or stable location (i.e., remote center of motion) to avoid stresson the skin of the patient. During set-up, any of the joints J0-J10 maymove. During surgery, the joints J6-10 may move subject to hardware orsafety limitations on position, velocity, acceleration, and/or torque.The surgical tool 220 may include none, one, or more (e.g., three)joints, such as a joint for tool rotation plus any number of additionaljoints (e.g., wrists, rotation about a longitudinal axis, or other typeof motion). Any number of degrees of freedom may be provided, such asthe three degrees from the joints J6-8 and none, one, or more degreesfrom the surgical tool 220.

In the example of FIG. 2 , the surgical tool includes joints J9 and J10,providing two DOF. In combination with the joints J6-8 in teleoperation,5 DOF is provided. In other embodiments, 4 DOF is provided, such as withan endoscope without the articulation joint J10. Unlike instruments withfully articulated wrists, the example of FIG. 2 (e.g., a staplinginstrument) may only have a single wrist joint J10. Thus, duringteleoperation, only five active joints (e.g., three arm joints J6-8 plusthe instrument shaft roll joint J9 and the wrist articulation joint J10)are available to position and orient the wrist and end effector 222.

In the example of FIG. 1 , the UID 116 is ungrounded. The UID 116 doesnot physically and/or directly connect to a base but is instead free tomove in 6 DOF in a Cartesian or other user command sensor space. The UID116 is ungrounded. The term “ungrounded” is intended to refer toimplementations where, for example, both UIDs are neither mechanicallynor kinematically constrained with respect to the user console. The UID116 does not include a structure that limits movement and/or structurefor directional haptic feedback. A wire may or may not flexibly connectto the UID 116. Given that, in general, a robot manipulator with fiveactuated joints J6-10 for teleoperation cannot achieve arbitrarypositions in 6D, 6D pose or commands from the UID 116 are projected intoa 4D or 5D subspace, which is reachable by the robot end effector 222.

FIG. 3 is a flow chart diagram of one embodiment of a method forteleoperation of a surgical robotic system. The method accounts for anylimited DOF of a tool or surgical instrument. Projection of the userinput or command to a subspace for the surgical instrument 220 providesa projected command to be used to solve for robotic control or movementof a robotic arm 122 and surgical tool 220. To address the limited DOFof the tool 220, projection is performed to convert the command tocontrol in the limited DOF.

The method of FIG. 3 is implemented by a control processor, such as thecontrol tower 130, computer, workstation, sever, or another processorperforming act 310. Any computer of the surgical robotic system 100 maybe used. A user interface provides the movement commands from the userreceived in act 300. The robotic arm 122 and/or surgical tool 220 aremoved using the instructions or control from the control processor inact 320. Other devices may perform and/or be used in any of the acts.

The acts are performed in the order shown or other orders. The variousacts 312 and 314 that are part of the solving of act 310 may beperformed in any order and/or simultaneously.

Additional, different, or fewer acts may be used. For example, act 322is not provided. As another example, the movement commands are from aprogrammed or processor determined sequence (e.g., operation template),so act 300 is not provided. In another example, acts for initiallypositioning the surgical tool 220 in the patient, planning surgery,and/or removing the surgical tool 220 from the patient may be provided.

In act 300, the control processor receives a user input command to movethe surgical tool 220 through movement of the robotic arm 122 and/orsurgical tool 220 of the robotic arm 122 during the teleoperation. Theuser input is received by the control processor from the user console110, such as the UIDs 116, via wireless or wired interface. In otherembodiments, the user inputs are received by loading from memory ortransmission over a computer network.

In preparation for teleoperation, the user sits down at the surgeonconsole 110. After positioning of the robot arm 122 for teleoperation,one or more joints are locked in place with a fixed remote center ofmotion (RCM) at the patient skin or incision entry point. For example,joints J0-J5 (see FIG. 2 ) are locked. The locking is by a brake and/oravoiding energizing the motors for the joints. These joints remainlocked during teleoperation.

During teleoperation, the user enters commands to move the end effector222 of the surgical instrument 220. The commands are for motion of theend effector 222. Different commands may be provided for differentmovements. The commands may be for movement of other parts of thesurgical instrument 220. A change in pose of the end effector 222 may beentered by sensing a pose or position and orientation or by sensing thechange. These commands may not be for movement of particular joints. Thecontrol processor is to convert the movement commands of the endeffector 222 or user input to controls of particular joints of therobotic arm 122 and/or surgical tool 220.

In one embodiment, user motion is tracked using a sensor. For example,the user holds a device, such as a pen or the UID 116. A magneticposition sensor and/or inertial measurement unit may be used todetermine pose and/or change in pose of the pen or UID 116. As anotherexample, the user holds a marker with a structure allowing for visualtracking, such as optical patterns or structures on one or more parts ofthe marker. A stereo camera and/or depth camera tracks the motion of themarker. Other user input devices may be used.

The user inputs are in 6 DOF. Translation along and rotation about allof three orthogonal axes is provided. The user inputs may be for controlof a surgical instrument 220 or end effector with fewer than six DOF,such as four or five DOF (e.g., translation along three axes butrotation along one or two axes). Alternatively, the user inputs are inless than 6 DOF but more DOF than provided by the robotic arm 122 andsurgical instrument 220.

The user inputs with six DOF may be for controlling movement for fewerthan six DOF. Five, four, or fewer active joints may be provided. FIG. 2shows the robotic arm 122 and the surgical tool 220 providing five DOF,such as where the surgical tool 220 is a stapler. The active jointsinclude three joints on the robotic arm 122—spherical rotation joint J6,spherical pitch joint J7, and tool translation joint J8. The activejoints include two joints on the surgical tool 220—rotation at joint J9and articulation joint J10 as a wrist. In an example of four DOF, theactive joints include three joints on the robotic arm 122—sphericalrotation joint J6, spherical pitch joint J7, and tool translation jointJ8—and one active joint on the surgical tool—rotation at joint J9. Otheractive joint arrangements may be used, such as providing two or fewerDOF on the robotic arm during teleoperation.

The DOF of the user inputs or commands is greater than the DOF ofmovement of the end effector 222. For example, the tool 220 has alimited degree of freedom, such as four or five DOF in combination ofthe tool 220 with the robotic arm 122 during teleoperation. The userinputs and corresponding UIDs 116 have six DOF. This may result inreceiving user inptus with rotation and/or translation of the tool aboutor along an axis where the tool is not rotatable about or translatablealong that axis when all other joints are stationary. In the example ofFIG. 2 , the tool may be rotatable only about two axes by rotation jointJ9 and wrist articulation joint J10, yet the user inputs may includerotation of the end effector 222 about all or any of three axes.

The end effector 222 may be moveable in six DOF using a combination ofjoints but cannot move in at least one DOF where the command is to moveonly in that DOF. During teleoperation, the remote center of motion(RCM) is a point on the vector pointing along the instrument shafttowards the end-effector 222 that is constrained to remain stationary.This is achieved through the manipulator mechanical design (via thespherical pitch joint design) or by control of the joints. The RCMconstraint is satisfied independently of the command for the sphericalarm joints (J6-J10) to move the end effector 222 during teleoperation.

In one embodiment, the end effector 222 is treated as being at thearticulation joint J10. The frame of reference for the end effector iscoincident with the articulation joint frame. FIG. 4 shows these framesof reference along the tool 220, including the RCM frame of referencewhere the RCM is to remain stationary along a plane tangential to theskin of the patient at the access point into the patient. The rotationand translation of the end effector 222 are represented as a tool rotatejoint frame and an articulation joint frame. The rotate joint frameincludes X, Y, and Z rotation axes, and the articulation joint frameincludes X, Y, and Z articulation axes. The end effector frame forteleoperation is specified to be coincident with the articulation frame,which is aligned with the rotation joint frame (e.g., Y rotate is alongbut an inverse of Z articulation, X rotate is along but an inverse of Yarticulation, and Z rotate is along and not an inverse of Xarticulation). Other arrangements or frames of reference may be used.

In act 310, the control processor projects the user input to a lessernumber of degrees of freedom than the six degrees of freedom of the UID116. The user inputs a pose or change in pose using the UID 116. Thisuser input for control of the end effector 222 has six DOF, so the poseor change in pose with 6 DOF is converted to a pose or change in posefor which the surgical tool is capable, such as pose or change with 5DOF.

In one embodiment, the 6D pose command for the end effector 222 at agiven time is provided. The pose for the current time may be computedfrom the pose from or for the end effector pose command at the previoustime step and the motion of the UID 116 between the previous and currenttime step. Alternatively, the 6D pose command for the current time isbased on a current pose of the UID 116 or a measure of the changewithout reference to an earlier pose.

Given that the UID 116 is ungrounded, the limitations on the robotactuation may render it infeasible for the robot end effector 222 toachieve the motion of the human hand within or pose for a given timestep. For example, rotation about one axis with no rotation about anyother axis may not be possible. The projection converts the user posecommand into realizable motion or pose. A pose command for the endeffector 222 is available that, after projection, is generallyachievable by the robotic arm 122 and surgical instrument 220.

In one embodiment, the projection is from the pose with 6 DOF to a posewith only 4 or 5 DOF. Rotation, translation, or both rotation andtranslation may be limited. In the example used herein (see FIG. 2 ),the projection is from 6 DOF to 5 DOF where rotation about one axisalone is not available. The projection is from 3 DOF of rotation to 2DOF of rotation while the 3 DOF for translation is maintained or notaltered in the projection. In alternative embodiments, it may betranslation along one axis alone not being available. For example, achange in position command with 3 DOF is projected onto a 2D subspace,and the full orientation command (e.g., 3 DOF) is maintained orunaltered. The delta position command is projected onto the X rotate andZ rotate axes. This would result in the user being unable toinstantaneously produce linear motion along the Yrotate axis, however,the user would have full orientation control. Other axes may be used. Inthe embodiment for projection of 6 DOF to 5 DOF where the missing DOF isrotation about one axis, the 3 DOF for translation are maintained andthe 3 DOF of rotation are projected to 2 DOF. The rotation about themissing DOF is projected onto the rotations about the remaining axes.

In one approach, the command end effector orientation are represented asa rotation matrix. Other representations may be used. The position androtation are defined with respect to a base frame at a given time t.Other frames of reference may be used. The position is denoted asp_(ee-base)(t)∈R^(3×1), and the rotation is denoted as R_(ee)^(base)(t)∈R^(3×3), where ee designates the end effector and basedesignates the base frame of reference. The orientation of the toolrotate joint frame (see FIG. 4 ) with respect to the base frame at timet is denoted as R_(rotate) ^(base)(t), where rotate designates the toolrotate joint frame. Other definitions and/or frames of reference may beused.

The projection is from 6 DOF to 5 DOF or 3 DOF for rotation (ortranslation) to 2 DOF for rotation (or translation. The translation orrotation in 3 DOF is maintained or not changed in the projection.

In one embodiment, given the kinematics of the spherical arm of therobotic arm 122 of FIGS. 2 and 4 , it is not possible for the endeffector 222 to instantaneously rotate about the Y articulation or Xrotation axis while maintaining a fixed position and fixed orientationin the remaining axes. To handle this, the change in end effectorcommand (delta end effector command) from the previous time step to thecurrent time step is projected onto a 5D subspace. The delta endeffector orientation command is projected onto the Z rotate axis, whichis aligned with the tool rotation joint axis, and Z articulation axis orinverse to the Yrotate axis, which is aligned with the wristarticulation joint axis. The surgical instrument 220 is able to producerotations instantaneously about these Y and Z rotate axes. The delta inrotation between the end effector orientation command at time t+1 andtime t represented in end effector coordinates is given by:

ΔR _(ee(t+1)) ^(ee(t))=(R _(ee) ^(base)(t))^(T) *R _(ee) ^(base)(t+1)

where T is the transpose. The change in rotation about three axes in anend effector frame of reference (e.g., at the articulation joint J10) isprovided by the rotation matrices at different times.

The rotation of the user input for rotation of the end effector 222 isprojected from rotation about three axes to rotation about two of theaxes. Any projection function, such as a linear or non-linear mapping,may be used. In one embodiment, the rotation is removed. In otherembodiments, the projection is converted with more than mere removal.For example, the projection uses a function relating rotation about oneaxis with rotation about one or more other axes. A dot product ofrotation about the axis to be removed with a change in rotation of theend effector is multiplied by the rotation about the axis to be removed.The result is subtracted from the rotation matrix for the user input.

In one embodiment, the projection uses conversion from the rotationmatrix to an axis-angle representation. The rotation matrixrepresentation is converted to an axis-angle representation. Theprojection is performed using the axis-angle representation. A result ofthe projection is converted back to the rotation matrix representation.For this approach, the delta end effector orientation is converted intothe axis-angle representation Δθ_(ee(t+1)) ^(ee(t))∈R^(3×1). Thisaxis-angle representation is projected onto the Y and Z axes of the toolrotate joint frame. For example, the following equation is used:

(Δθ_(ee(t+1)) ^(ee(t)))_(projected)=Δθ_(ee(t+1)) ^(ee(t))−(Δθ_(ee(t+1))^(ee(t)) ·X _(rotate-in-ee)(t))*X _(rotate-in-ee)(t)

where (·) is the dot product operator. X_(rotate-in-ee) is the X axis ofthe tool rotate frame expressed in end effector frame coordinates and iscomputed using the following equation:

R _(rotate) ^(ee)(t)=[X _(rotate-in-ee) Y _(rotate-in-ee) Z_(rotate-in-ee)]=(R _(ee) ^(base)(t))^(I) *R _(rotate) ^(base)(t)

The projected axis-angle representation (Δθ_(ee(t+1))^(ee(t)))_(projected) is converted back into rotation matrixrepresentation as (ΔR_(ee(t+1)) ^(ee(t)))_(projected). Other conversionsor representations or calculation in the rotation matrix may be used.

The projection to the reduced dimension subspace may be used alone. Inother embodiments, the projection includes consideration for otherphysical relationships of the robotic arm 122 and surgical instrument220. In one embodiment, the projection includes projection to thereduced dimension subspace (e.g., 6 DOF to 5 DOF) of the end effectorcommand as well as a projection of a change in rotation due to couplingbetween translation and rotation. In movement of the end effector 222,the translation and rotation provided by the robotic arm 122 andsurgical instrument 220 are coupled. The rotation and translation aboutone axis (e.g., the rotation axis being removed by projection) may becoupled so that translation along the axis affects rotation. Therotation and translation along other axes and/or between axes may becoupled. The projection may include consideration of one or more ofthese relationships.

The remote center of motion limitation on movement of the end effector222 impacts or causes the coupling. The projection may include asimilarity transform to account for the remote-center-of-motion of therobotic arm 122 with coupling of translation and rotation.

In one embodiment, the projection to the reduced dimension subspacenulls out orientation or change in orientation along the X_(rotate)axis. The change in position is unchanged by the projection. In order toachieve a desired delta in commanded end effector position along theY_(rotate) direction, a delta in orientation occurs about the X_(rotate)axis as these motions are coupled and cannot be controlledindependently. This delta in orientation is computed given theconstraint that the RCM position is fixed.

The orientation and position of the RCM frame with respect to the baseframe are denoted as p_(rcm-base)(t) and R_(rcm) ^(base)(t),respectively. The commanded end effector poses with respect to the RCMframe at times t and t+1 and the orientation of the end effector framewith respect to the RCM frame at time t are computed as:

p _(rcm-ee)(t)=(R _(rcm) ^(base)(t))^(T)*(p _(ee-base)(t)−p_(rcm-base)(t))

p _(rcm-ee)(t+1)=(R _(rcm) ^(base)(t))^(T)*(p _(ee-base)(t+1)−p_(rcm-base)(t))

R _(ee) ^(rcm)(t)=(R _(rcm) ^(base)(t))^(T) R _(ee) ^(base)(t)

The rotation or change in rotation for time t+1 is determined as part ofthe projection operation. The normalized unit vectors corresponding top_(rcm-ee)(t) and p_(rcm-ee)(t+1) are denoted as {circumflex over(ν)}_(rcm-ee)(t) and {circumflex over (ν)}_(rcm-ee)(t+1), respectively.The rotation to transform the vector {circumflex over (ν)}_(rcm-ee)(t)into {circumflex over (v)}_(rcm-ee)(t+1) is computed using Rodrigues'rotation formula. This rotation, in matrix representation, from thetranslation is denoted as ΔR_(ee-rcm-constraint). This rotation isconverted into the end effector frame using the similarity transform, asrepresented by:

ΔR _(ee-constraint(t+1)) ^(ee-constraint(t))=(R _(ee) ^(rcm)(t))^(T) *ΔR_(ee-rcm-constraint) *R _(ee) ^(rcm)(t)

The result is a constrained rotation in the end effector frame. The axisangle representation of this rotation, denoted asΔθ_(ee-constraint(t+1)) ^(ee-constraint)(t), is projected onto theX_(rotate) axis as follows:

(Δθ_(ee-constraint(t+1))^(ee-constraint(t)))_(projected)=(Δθ_(ee-constraint(t+1))^(ee-constraint(t)) ·X _(rotate_in_ee)(t))*X _(rotate-in-ee)(t)

This rotation is converted to the rotation matrix representation,denoted as (ΔR_(ee-constraint(t+1))^(ee-constraint(t))(t)_(projected. Other functions may be used to determine the rotation or change in rotation due to coupling or other physical constraint in movement of the robotic arm 122 and surgical instrument 220.)

This projected change in rotation due to coupling is combined with theprojection to reduce the DOF. Any combination may be used, such as aweighting, summation, or multiplication. The final projected user inputis a function of both the change in rotation due to the coupling and theprojection to the lesser number of DOF. In one embodiment, the functionis a multiplication of the change in rotation due to the coupling, theprojection to the lesser number, and the rotation about the three axesof the original user input for motion of the end effector. To producethe final projected end effector orientation command (e.g., change inorientation), the delta rotation command for DOF reduction and deltarotation due to the coupling between the linear motion and rotationabout the X_(rotate) axis are combined as follows:

(ΔR _(ee) ^(base)(t+1))_(projected) =R ₃₃ ^(base)(t)*(ΔR_(ee-constraint(t+1)) ^(ee-constraint(t)))_(projected)*(ΔR _(ee(t+1))^(ee(t)))_(projected)

Other combinations may be used to determine the change in rotation fromprojection. The result is the user input for end effector pose or changein pose projected to the lower dimensional subspace to account forlimitations of the rotation. The result is a projected user input forend effector pose or change in pose. The projection may be of therotation of the pose while the translation is used without change.

It is assumed that the delta rotations are small given a real timeteleoperation scenario due to frequent sampling or measurement of theuser interface device 116. If the delta rotation is large, combiningthese delta rotations from dimensionality reduction and coupling may usean additional transformation to transform the delta rotation fromcoupling to be with respect to the frame produced by applying the deltarotation for dimensionality reduction. Without the additionaltransformation, both delta rotations are defined with respect to the endeffector frame at time t.

The resulting projected user input or command is used for controllingthe end effector 222 instead of the user input before projection. Theend effector 222 is to be controlled based on the pose provided afterprojection.

In act 320, the control processor solves for joint motion of the roboticarm 122 and surgical tool 220 from the projected user command. Thecommand, as projected rather than without projection, for motion or poseof the end effector is used to determine which joints to move, by howmuch, and in which direction. The receipt of the user input and theprojection occur in a command space or the domain of the user input forcontrol of the end effector 222. The projection is performed prior tosolving for the joint motions. The solution then solves for the jointmotions from the projected user command (e.g., the projected endeffector pose or change in pose). The control processor solves byconverting from the command space for the end effector 222 to a jointspace for joints of the surgical tool 220 and robotic arm 122. Thecontrol processor translates the movement commands from the user tomovement of the joints.

The controller or another processor solves for joint motion from theprojected user command with inverse kinematics. The control processorsolves for motion by the robotic arm 122 and/or the surgical tool 220 ofthe robotic arm 122 with an iterative solution. An iterative inversekinematic solution is found. The control process may be input of theuser inputs, iterative solution using inverse kinematics with a giventermination check for the iterations, and output of a final result ofjoint commands. The inverse kinematics may incorporate limits onposition, velocity, torque, and/or acceleration on the motion of the endeffector and/or joints. The inverse kinematics is an optimizationfunction, such as a minimization. For example, a difference between theprojected change in pose and a change in pose resulting from jointpositions of the surgical robotic system is minimized in act 312. Theminimization provides for the change in the joint position given thechange in pose. Other optimizations may be used.

In one embodiment, the inverse kinematics is performed as a least squareminimization. Other minimizations may be used. The minimization issolved from in a control frame different than the end effectorcoordinate system. For example, the control frame is a frame ofreference for the robotic arm 122, such as a coordinate system based onthe joint JO or a base of the robotic arm 122.

In act 322, the control processor causes movement of the robotic arm 122and/or the surgical tool 220. The output movement commands for theactive joints (e.g., joints J6-10 of FIG. 2 ) during teleoperation causethe joints to change position at the velocity and/or acceleration. Theresults from the inverse kinematics control the movement of the joints.The joint motion avoids rotation of the surgical tool 220 in ways thatare not feasible. The solution from the inverse kinematics is used tomove the surgical tool 220 by operation of joints of the surgical tool220 and/or joints of the robotic arm 122 holding the surgical tool 220.The solved for changes in joint position control the surgical roboticsystem.

The expected behavior of the robotic arm 122 and surgical instrument 220given the projection is that the robot end effector 222 will track theposition command from the user input device 116, but the orientationcommand is projected onto a plane (i.e., from X, Y, Z (3D) to Y, Z(2D)). For example, if at a given time, the delta in orientation of theuser interface device 116 and corresponding user input is about theX_(rotate) axis and the user interface device 116 position isstationary, the end effector would remain stationary as the deltaorientation command is projected into the plane that is orthogonal toX_(rotate). With only rotation in X, the rotation does not occur due tothe projection to a plane without an X rotation component. The user isnot able to instantaneously produce an end effector rotation about theX_(rotate) axis although the ungrounded user interface device 116 isfree to rotate about that axis. The projection results a change in themapping between the physical axes of the user interface device 116 andthe end effector orientations. If the failure to rotate in thissituation is undesirable to users, the user may be trained to learn torotate the user interface device 116 predominantly in the projectedsubspace (e.g., rotation in Y and Z). Additionally or alternatively, theend effector orientation command may be used to attempt to keep the userinterface device axes and end effector axes aligned.

FIG. 5 shows a block diagram of one embodiment of a surgical roboticsystem for medical teleoperation. The user inputs from the user input508 are projected to a lower dimensionality subspace. The robotic systemis controlled based on the user command as projected. The systemperforms the method of FIG. 3 or another method.

The surgical robotic system includes one or more robot arms 122 withcorresponding surgical instruments 220 or other types of instrumentsconnected with the robot arms 122, a controller 502, and a memory 504.The user console 110 is represented or included as part of the surgicalrobot system but may be positioned remotely from or locally to the robotarm 122. Additional, different, or fewer components may be provided. Forexample, the robot arm 122, surgical instrument 220, and/or user console110 are not provided.

The robotic arms 122 each include one or more links and joints. Thejoints may be pitch or roll joints. A tool drive and cannula forreceiving and guiding a surgical tool may be provided on each of therobotic arms 122. Different combinations of links and joints may defineor form different parts of the robotic arms 122, such as different partshaving different degrees or types of movement (e.g., translation and/orrotation). Any now known or later develop robotic arm 122 with motors,sensors, links, joints, controllers, surgical instruments, and/or otherstructure may be used.

One or more robotic arms are provided. For example, three or fourrobotic arms 122 are provided. The robotic arms 122 mount to a table,such as a base of an operating table. Alternatively, cart, floor,ceiling, or other mounts may be used. The robotic arms 122 include acable or wireless transceiver for communication with the processor 206or an intermediary (e.g., control tower 130).

The robotic surgical instruments 220 are one or more graspers,retractors, scalpels, endoscopes, staplers, scissors, or other surgicaldevice for manipulating tissue of the patient. The tissue manipulationmay be direct, such as cutting or grasping. The tissue manipulation maybe indirect, such as an endoscope pressing or contacting tissue asguided to image or view an interior portion of the patient. Different orthe same type of instruments 220 may be mounted to different ones of therobot arms 122. For example, two robot arms 122 may have graspers, athird robot arm 122 may have a scalpel, and a fourth robot arm 122 mayhave an endoscope.

The robotic surgical instruments 220 connect to the distal ends of therobot arms 122 but may connect at other locations. The connectionprovides a drive so that the tool may be operated, such as closing agrasper or scissors and for operating joints of the surgical instrument220.

One or more of the robotic surgical instruments 220 has limited motion.The surgical instrument 220 in combination with the robot arm 122 mayhave fewer than six DOF, such as having four or five DOF. For example,the robot arm 122 provides three joints while the surgical instrument220 is limited to rotation about one axis or two axes. An endoscope asthe surgical tool 220 may provide for just rotation about the long axisof the instrument 220 without rotation about two other orthogonal axes.A stapler as the surgical tool 220 may provide for rotation about thelong axis and rotation about another axis for one joint withoutproviding for rotation about a third axis. The robot arm 122 may allowfor full 6 DOF. The robot arm 122 may have some movement locked duringteleoperation. As a result, the surgical instrument 220 or the surgicalinstrument 220 in combination with the robot arm 122 may not be able torotate and/or translate just about one or more axes. For example, theend effector 222 of the surgical instrument 220 as connected to therobotic arm 122 cannot rotate about one axis while not rotating abouttwo other axes.

The user console 110 is a graphics user interface for interaction of thesurgeon with the surgical robot system, such as with a processor (e.g.,controller 502 or another controller) for controlling the robotic arms122. The user interface includes a user input 508 and a display 118. Theuser input 508 and/or the display 118 are provided at the user console110 and/or control tower 130 but may be at other locations.

The user input 508 is a button, a keyboard, a rocker, a joy stick, atrackball, a voice recognition circuit, a mouse, a touch pad, a touchscreen, sliders, switches, UID 116, foot pedal 114, combinationsthereof, or any other input device for inputting to the surgical robot.The user input 508 may be a sensor or sensors for detecting eye movementand/or blinking. In yet other embodiments, the user input 508 is amicrophone for voice-based input. The user input 508 has a greaterdegree of freedom of motion than the end effector 222. For example, theUID 116 is untethered or ungrounded, so has 6 DOF. The end effector 222may be limited to not rotate or translate about one or more axes whereno motion is provided for other axes.

The display 118 is a monitor, liquid crystal display (LCD), projector,plasma display, CRT, printer, or other now known or later developeddevice for outputting visual information. In an alternative embodiment,the display 118 is a head mounted display. A speaker for output of audioinformation may be provided instead of or in addition to the display118.

The controller 502 is a controller that drives and/or models the roboticarms 122 and/or surgical instruments 220. The controller 502 is ageneral processor, central processing unit, control processor, graphicsprocessor, graphics processing unit, digital signal processor,application specific integrated circuit, field programmable gate array,digital circuit, analog circuit, artificial intelligence processor,combinations thereof, or other now known or later developed device fortranslating user inputs to joint commands for the robot arm 122 and/orsurgical instrument 220. The controller 502 is a single device ormultiple devices operating in serial, parallel, or separately. Thecontroller 502 may be a main processor of a computer, such as a laptop,server, workstation, or desktop computer, or may be a processor forhandling some tasks in a larger system. Based on hardware, software,firmware, or combinations thereof, the controller 502 is configured toimplement instructions or perform acts.

The controller 502 is configured to project a user input from the userinput 508 (e.g., UID 116) for motion of the end effector 222. The userinput may include unavailable movement, such as including unavailablerotation about an axis. The unavailable movement is projected, such asprojecting rotation about one axis to rotations about just the two otheraxes. The projection may include other constraints, such as projectionof the user input to one change in rotation due to DOF reduction andprojection of the user input to another change in rotation due to acoupling between linear motion and rotation. The resulting projecteduser command is a function of both changes. The projection is providedfor isolated movement to the unavailable and/or provided even when themovement is not isolated to the unavailable (e.g., user input is fortranslation along two axes and rotation about two axes where the robotarm 122 and surgical instrument 220 cannot rotation about one of the twoaxes when no other motion is provided).

The controller 502 is configured to solve for joint motion of therobotic arm and the surgical instrument with inverse kinematics from theprojected user command. The solution is provided for part of or duringmedical teleoperation on a patient. The projected motion of the surgicalinstrument 220 is used to solve for joint commands to move the joints ofthe robot arm 122 and/or surgical instrument 220. These joint commandsor motion are solved for in response to projection of user input of amove command. The user inputs a command to move (e.g., inputs a changeor inputs a pose) the end effector 222 of the surgical instrument 220.The controller 502 is configured to project and then solve for themotion of the surgical instrument 220 through operation of the joints toprovide the motion as projected.

The controller 502 is configured to solve for the motion with inversekinematics. For example, a least square minimization of a differencebetween the motion of the surgical instrument 220 from the joint motionsand the projected user command is used. Other optimizations relating thejoint commands to the projections of end effector movement commandsinput by the user may be used.

The controller 502 is configured to control the robot arm 122 andsurgical tool 220. Based on the solution from the inverse kinematics,one or more joints are moved in response to user inputs. The iterativeinverse kinematic solution controls the joints.

The memory 504 or another memory is a non-transitory computer readablestorage medium storing data representing instructions executable by theprogrammed controller 502. The instructions for implementing theprocesses, methods and/or techniques discussed herein are provided oncomputer-readable storage media or memories, such as a cache, buffer,RAM, removable media, hard drive or other computer readable storagemedia. Computer readable storage media include various types of volatileand nonvolatile storage media. The functions, acts or tasks illustratedin the figures or described herein are executed in response to one ormore sets of instructions stored in or on computer readable storagemedia. The functions, acts or tasks are independent of the particulartype of instructions set, storage media, processor or processingstrategy and may be performed by software, hardware, integratedcircuits, firmware, micro code and the like, operating alone, or incombination. Likewise, processing strategies may includemultiprocessing, multitasking, parallel processing, and the like.

In one embodiment, the instructions are stored on a removable mediadevice for reading by local or remote systems. In other embodiments, theinstructions are stored in a remote location for transfer through acomputer network or over telephone lines. In yet other embodiments, theinstructions are stored within a given computer, CPU, GPU, or system.

While the invention has been described above by reference to variousembodiments, it should be understood that many changes and modificationscan be made without departing from the scope of the invention. It istherefore intended that the foregoing detailed description be regardedas illustrative rather than limiting, and that it be understood that itis the following claims, including all equivalents, that are intended todefine the spirit and scope of this invention.

I(We) claim:
 1. A method for teleoperation of a surgical robotic system,the method comprising: receiving a user input to move a surgical toolcoupled to a robotic arm, the user input having six degrees of freedomwhere the surgical tool has a lesser number of degrees of freedom;projecting the user input to the lesser number of degrees of freedom;solving for joint motion of the robotic arm and surgical tool from theprojected user input with inverse kinematics; and moving the robotic armand/or surgical tool based on a solution from the solving.
 2. The methodof claim 1 wherein receiving comprises receiving the user input withrotation in three degrees of freedom where the surgical tool isrotatable in two degrees of freedom; and wherein projecting comprisesprojecting the user input from the rotation in the three degrees offreedom to rotation in the two degrees of freedom.
 3. The method ofclaim 1 wherein receiving comprises receiving the user input from ahandheld user interface device that is ungrounded, and wherein the userinput is for an end effector of the surgical tool.
 4. The method ofclaim 1 wherein projecting comprises maintaining the translation inthree degrees of freedom and reducing the rotation from three degrees offreedom to the less than three degrees of freedom.
 5. The method ofclaim 1 wherein receiving comprises receiving a first pose or firstchange in pose as the user input, and wherein projecting comprisesconverting the pose or change in pose to a second pose or second changein pose for which the surgical tool is capable.
 6. The method of claim 1wherein receiving and projecting occur in a command space for an endeffector of the surgical tool prior to solving, and wherein solvingcomprises converting from the command space for the end effector to ajoint space for joints of the surgical tool and robotic arm.
 7. Themethod of claim 1 wherein projecting comprises projecting the user inputfor the rotation about first, second, and third axes as the three axesto rotation about just the first and second axes as the lesser number bya subtraction of a multiplication of (1) a dot product of a rotationabout the third axis with a change in rotation of the end effector ofthe user input by (2) the rotation about the third axis.
 8. The methodof claim 7 wherein the user input is in a rotation matrixrepresentation, and wherein projecting comprises converting the rotationmatrix representation to an axis-angle representation, projecting in theaxis-angle representation, and converting a result of the projectingback to the rotation matrix representation.
 9. The method of claim 1wherein projecting comprises projecting a change in rotation due tocoupling between the translation and the rotation.
 10. The method ofclaim 9 wherein projecting the change in the rotation due to thecoupling comprises projecting with a similarity transform accounting fora remote-center-of-motion of the robotic arm.
 11. The method of claim 9wherein the projected user input is a function of the change in rotationdue to the coupling and the projection to the lesser number of degreesof freedom.
 12. The method of claim 11 wherein the function is amultiplication of the change in rotation due to the coupling, theprojection to the lesser number, and the rotation about the three axes.13. The method of claim 1 wherein solving comprises solving as aminimization of a least square.
 14. A method for accounting for alimited degree of freedom of a tool in a surgical robotic system, themethod comprising: projecting a first pose from an ungrounded userinterface with six degrees of freedom to a second pose of an endeffector of a surgical tool held by a robotic arm where the second posehas only four or five degrees of freedom; and controlling the endeffector based on the second pose.
 15. The method of claim 14 whereinprojecting comprises projecting with an axis-angle representation by asubtraction of a multiplication of (1) a dot product of a rotation aboutor translation on a first axis with a change in rotation or translationof the end effector by (2) the rotation about or translation on thefirst axis.
 16. The method of claim 14 wherein projecting comprisesprojecting with a change in rotation or translation due to a couplingbetween the rotation and translation about a first axis.
 17. The methodof claim 14 further comprising receiving a user input command from auser interface, the first pose being of the end effector provided as theuser input command.
 18. The method of claim 14 wherein the second posehas only five degrees of freedom, a missing degree of freedom of thefive degrees as compared to the six degrees being rotation about a firstaxis, and wherein projecting comprises maintaining three degrees offreedom in translation and projecting the rotation about the axis torotation about second and third axes.
 19. A surgical robotic systemcomprising: a robotic arm; a surgical instrument mountable to therobotic arm having an end effector where rotation about one axis iscoupled to rotation about another axis; a user interface device havingthree degrees of freedom in rotation; and a controller configured toproject a user command from the user interface device for rotation aboutthe one axis to rotations about the one axis and the other axis.
 20. Thesurgical robotic system of claim 19 wherein the controller is configuredto project the user command to a first change in rotation and to projectthe user command to a second change in rotation due to the coupling, thecoupling being between linear motion and rotation, a projected usercommand being a function of the first and second changes, and thecontroller being configured to solve for joint motion of the robotic armand the surgical instrument with inverse kinematics from the projecteduser command.